from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.870	0.018	0.0	0.017	-1	1	0.663	1.826	0.036	9.237	0.185	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.870	0.018	0.0	0.017	-1	1	0.663	0.024	0.004	717.507	127.879	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.313	0.006	0.0	0.313	-1	1	1.000	1.826	0.036	0.171	0.005	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.313	0.006	0.0	0.313	-1	1	1.000	0.024	0.004	13.314	2.387	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.947	0.022	0.0	0.017	-1	5	0.757	2.793	0.086	6.067	0.186	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.947	0.022	0.0	0.017	-1	5	0.757	0.024	0.003	698.855	78.883	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.280	0.008	0.0	0.280	-1	5	1.000	2.793	0.086	0.100	0.004	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.280	0.008	0.0	0.280	-1	5	1.000	0.024	0.003	11.546	1.348	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.111	0.020	0.0	0.015	1	100	0.882	1.855	0.003	8.147	0.016	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.111	0.020	0.0	0.015	1	100	0.882	0.019	0.000	795.600	4.333	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.270	0.005	0.0	0.270	1	100	1.000	1.855	0.003	0.146	0.003	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.270	0.005	0.0	0.270	1	100	1.000	0.019	0.000	14.231	0.278	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.088	0.114	0.0	0.015	-1	100	0.882	2.675	0.053	5.641	0.120	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.088	0.114	0.0	0.015	-1	100	0.882	0.024	0.002	628.964	49.290	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.277	0.007	0.0	0.277	-1	100	1.000	2.675	0.053	0.103	0.003	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.277	0.007	0.0	0.277	-1	100	1.000	0.024	0.002	11.532	0.949	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.079	0.024	0.0	0.015	1	5	0.757	1.846	0.004	8.167	0.021	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.079	0.024	0.0	0.015	1	5	0.757	0.019	0.000	795.407	5.656	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.273	0.008	0.0	0.273	1	5	1.000	1.846	0.004	0.148	0.004	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.273	0.008	0.0	0.273	1	5	1.000	0.019	0.000	14.391	0.442	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.346	0.006	0.0	0.015	1	1	0.663	1.072	0.016	14.316	0.214	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	15.346	0.006	0.0	0.015	1	1	0.663	0.018	0.000	847.045	22.362	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.281	0.005	0.0	0.281	1	1	1.000	1.072	0.016	0.262	0.006	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.281	0.005	0.0	0.281	1	1	1.000	0.018	0.000	15.501	0.498	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.264	0.006	0.0	0.003	-1	1	0.896	1.614	0.031	2.023	0.039	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.264	0.006	0.0	0.003	-1	1	0.896	0.007	0.006	439.255	330.086	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	1	1.000	1.614	0.031	0.148	0.004	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	1	1.000	0.007	0.006	32.121	24.144	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.596	0.006	0.0	0.004	-1	5	0.922	2.533	0.081	1.420	0.045	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.596	0.006	0.0	0.004	-1	5	0.922	0.009	0.003	409.400	116.938	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	5	1.000	2.533	0.081	0.094	0.003	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	5	1.000	0.009	0.003	27.178	7.778	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.651	0.013	0.0	0.004	1	100	0.929	1.885	0.010	1.937	0.012	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.651	0.013	0.0	0.004	1	100	0.929	0.003	0.000	1317.016	81.969	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	1	100	1.000	1.885	0.010	0.127	0.002	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	1	100	1.000	0.003	0.000	86.071	5.546	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.640	0.006	0.0	0.004	-1	100	0.929	2.554	0.026	1.425	0.015	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.640	0.006	0.0	0.004	-1	100	0.929	0.011	0.006	326.601	171.439	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	100	1.000	2.554	0.026	0.094	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	-1	100	1.000	0.011	0.006	21.462	11.271	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.596	0.006	0.0	0.004	1	5	0.922	1.871	0.004	1.922	0.005	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.596	0.006	0.0	0.004	1	5	0.922	0.003	0.000	1291.593	89.788	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	1	5	1.000	1.871	0.004	0.128	0.002	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.239	0.004	0.0	0.239	1	5	1.000	0.003	0.000	85.751	6.128	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.654	0.009	0.0	0.004	1	1	0.896	0.983	0.006	3.718	0.024	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.654	0.009	0.0	0.004	1	1	0.896	0.002	0.000	2181.815	214.882	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.237	0.005	0.0	0.237	1	1	1.000	0.983	0.006	0.241	0.005	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.237	0.005	0.0	0.237	1	1	1.000	0.002	0.000	141.646	14.203	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	115.150	0.000	0.0	0.115	-1	1	0.929	0.432	0.002	266.805	1.365	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	115.150	0.000	0.0	0.115	-1	1	0.929	0.003	0.001	42133.544	8153.326	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.557	0.070	0.0	2.557	-1	1	1.000	0.432	0.002	5.924	0.166	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.557	0.070	0.0	2.557	-1	1	1.000	0.003	0.001	935.588	182.870	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.007	0.000	0.0	0.119	-1	5	0.946	0.857	0.008	138.832	1.271	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.007	0.000	0.0	0.119	-1	5	0.946	0.003	0.000	35787.746	4756.210	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.568	0.092	0.0	2.568	-1	5	1.000	0.857	0.008	2.995	0.110	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.568	0.092	0.0	2.568	-1	5	1.000	0.003	0.000	772.150	106.247	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.545	0.000	0.0	0.123	1	100	0.951	4.883	0.016	25.098	0.085	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.545	0.000	0.0	0.123	1	100	0.951	0.004	0.000	33379.481	3794.500	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.554	0.052	0.0	2.554	1	100	1.000	4.883	0.016	0.523	0.011	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.554	0.052	0.0	2.554	1	100	1.000	0.004	0.000	695.643	80.351	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.302	0.000	0.0	0.129	-1	100	0.951	2.900	0.031	44.584	0.477	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.302	0.000	0.0	0.129	-1	100	0.951	0.005	0.001	24043.658	5318.609	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.567	0.100	0.0	2.567	-1	100	1.000	2.900	0.031	0.885	0.036	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.567	0.100	0.0	2.567	-1	100	1.000	0.005	0.001	477.366	107.229	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.416	0.000	0.0	0.136	1	5	0.946	1.459	0.005	93.491	0.331	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.416	0.000	0.0	0.136	1	5	0.946	0.001	0.000	93484.446	21364.237	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.559	0.062	0.0	2.559	1	5	1.000	1.459	0.005	1.754	0.043	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.559	0.062	0.0	2.559	1	5	1.000	0.001	0.000	1753.833	403.060	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	134.368	0.000	0.0	0.134	1	1	0.929	0.784	0.004	171.403	0.772	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	134.368	0.000	0.0	0.134	1	1	0.929	0.001	0.000	139792.483	29515.727	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.587	0.121	0.0	2.587	1	1	1.000	0.784	0.004	3.300	0.156	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.587	0.121	0.0	2.587	1	1	1.000	0.001	0.000	2691.725	582.185	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	1	0.891	0.021	0.001	1.798	0.110	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	1	0.891	0.002	0.000	15.504	2.297	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.021	0.001	0.244	0.016	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.108	0.315	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.033	0.000	0.0	0.000	-1	5	0.911	0.022	0.001	1.554	0.066	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.033	0.000	0.0	0.000	-1	5	0.911	0.002	0.000	16.752	2.424	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.022	0.001	0.212	0.011	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.286	0.338	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.052	0.000	0.0	0.000	1	100	0.894	0.030	0.000	1.762	0.010	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.052	0.000	0.0	0.000	1	100	0.894	0.001	0.000	87.525	18.379	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.030	0.000	0.157	0.006	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	7.805	1.669	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.053	0.000	0.0	0.000	-1	100	0.894	0.037	0.007	1.428	0.268	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.053	0.000	0.0	0.000	-1	100	0.894	0.002	0.000	27.305	6.922	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.037	0.007	0.125	0.024	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.000	2.382	0.607	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	5	0.911	0.021	0.001	1.795	0.072	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	5	0.911	0.001	0.000	66.203	13.686	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.021	0.001	0.242	0.011	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.941	1.860	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.891	0.019	0.000	1.947	0.017	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	1	0.891	0.001	0.000	65.322	13.836	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.019	0.000	0.262	0.006	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.796	1.873	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.447	0.017	300	0.002	0.00	0.824	0.108	0.0	4.123	0.157	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.447	0.017	300	0.002	0.00	0.824	0.017	0.0	26.761	1.145	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.370	0.015	300	0.000	0.37	1.000	0.108	0.0	3.412	0.135	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.370	0.015	300	0.000	0.37	1.000	0.017	0.0	22.143	0.979	See	See